Recently, the amount of data for image transmission has been increasing day by day, for example, under the influence of terrestrial digital television broadcasting that has been in service since 2011. The image resolution to be used generally has changed from analog resolution (720 pix×480 pix) used so far to full high-definition (HD) resolution (1920 pix×1080 pix).
Also in a monitoring camera used at various locations such as stations, airports, buildings or rivers, a high-definition CCD or CMOS sensor is used, and the resolution is remarkably improved from the conventional analog resolution.
Therefore, a high-quality image can be photographed and transmitted, but the transmission bandwidth required increases. In the case of transmitting a full HD image in an uncompressed manner, the transmission bandwidth of about 1.5 Gbps is required. In the transmission through a packet switching network, since even an optical fiber that is widely used as a current home broadband has a bandwidth of about 100 Mbps, the full HD image cannot be transmitted in an uncompressed manner.
Thus, it is necessary to compress the data to a transmittable bandwidth. Currently, H.264 and H.265 may be used as high-efficiency compression technology standards. These standards are international standards of moving image encoding.
In a compression technique, the transmission bandwidth and the image quality are in a trade-off relationship. In the case of high compression for low bandwidth transmission, a deterioration in image quality occurs. Accordingly, it is required to obtain a high-quality image even when the image is transmitted at a low bandwidth. Therefore, a super-resolution technique has been attracting attention to achieve a high resolution by digital image processing.
FIG. 20 is a schematic block diagram of an image transmission system having a general image processing device. As shown in FIG. 20, the image transmission system includes a video encoder 1 as a configuration of the transmission side. Further, the image transmission system includes a video decoder 2 and a super-resolution processing unit 3 as a configuration of the reception side.
In the example of FIG. 20, only the configuration used for transmission is illustrated in the image processing device on the transmission side, and only the configuration used for reception is illustrated in the image processing device on the reception side.
The video encoder 1 on the transmission side encodes an image inputted from an image input device such as a CCD (charge-coupled device) camera in a coding scheme (e.g., H.264), and outputs the encoded image data to a transmission path such as an IP network or a coaxial cable network.
The video decoder 2 on the reception side decodes the image data received through the transmission path in a scheme corresponding to the coding scheme in the video encoder 1.
The super-resolution processing unit 3 performs a super-resolution process on the decoded image to obtain an image having a higher resolution as an output image.
In the coding scheme such as H.264 or H.265, various coding tools have been defined. Accordingly, it is possible to improve the image quality by performing flexible adjustment according to the transmission bandwidth. Then, at the video decoder side, it is possible to improve the image quality by performing a super-resolution process on the image with less degradation in image quality.
However, for example, in the case of using an encoder capable of specifying only the bit rate and the image size, since fine adjustment of a compression process cannot be performed, a degradation in image quality occurs. Therefore, even if a super-resolution process is performed at the decoder side, an image quality improvement effect cannot be expected.
(Compression in Encoding Moving Image Data)
An improvement of the compression ratio in encoding the moving image will now be described. A moving image encoding method has been practically used by developing a moving image encoding system which is represented by MPEG (Moving Picture Experts Group). Further, in order to improve the compression efficiency, for example, there has been proposed a technology for improving the efficiency of orthogonal transformation, a quantization process and a variable length coding process of the next stage by using a difference value between a predicted value and a pixel value using intra prediction and inter prediction (see, e.g., Non-patent Document 1).
Information handled in moving image coding includes a combination of information (moving image information) on pixels constituting the image, information (header information) describing how to handle the moving image information, and motion vector information used for inter prediction.
Due to recent technological progress, the compression efficiency of the moving image information has been improved remarkably, but the improvement of the compression ratio of the header information and the motion vector information is not so high.
As the related art relating to an image transmission apparatus, there are the specifications of International Publication No. WO 2010/137323 “VIDEO ENCODER, VIDEO DECODER, VIDEO ENCODING METHOD, AND VIDEO DECODING METHOD” (Mitsubishi Electric Corporation, Patent Document 1), Japanese Patent No. 4245576 “IMAGE COMPRESSION/DECOMPRESSION METHOD, IMAGE COMPRESSION APPARATUS AND IMAGE DECOMPRESSION APPARATUS” (TOA Corporation, Patent Document 2), and Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG: Text of ISO/IEC 14496-10 Advanced Video Coding 3rd Edition (2004) (Non-patent Document 1).
Patent Document 1 discloses that a compressing unit of a video encoding device selects whether to transform and quantize a prediction error signal after reduction and transformation to create quantization coefficient data, and in the case of reduction, after inverse-quantizing and inverse-transforming the quantization coefficient data, an enlargement transformation is performed to create a decoded prediction error signal.
Patent Document 2 discloses an image compression and decompression method which includes converting other regions than a region designated as a significant region into reduced data, re-arranging a significant region and reduced data in an image region matched with the horizontal width of the significant region to generate reduced image data, performing motion-compensated prediction and encoding processing, restoring the original arrangement based on the header information after decompression processing at the time of restoration, and performing compensation processing on the reduced data to restore the image data.